API hooks collectively refer to techniques that bypass the call flow of a function to another location through specific manipulation with regard to a general API call process. FIGS. 1 and 2 illustrate a basic call structure, and an API filtering process based on a hook. As illustrated in FIG. 2, a hook function B intervenes in a process in which a target function C is called by an original function A. A hook layer that is introduced in order to fully realize the hook technique should not be influenced by a hook.
The ideal structure of a hook layer, such as that shown in FIG. 3, should provide support so that hook and substitution/filtering processing can be performed in connection with a plurality of functions. The hook layer itself should not be influenced by the hook. In order to support such a structure, a hook layer should not use a hook target API, or should clearly know addresses used to call hook target API original functions and selectively call a hook API and an original API as desired. When the hook layer does not use the hook target API, an operation inside the hook layer has nothing to do with an API hook. However, it is not easy to prevent the hook layer from using the hook target API. That is, the case in which the hook layer does not use the hook target API is possible only when there is no direct connection between the hook target API and a run time binary for a lower original function. Accordingly, since it is necessary to implement an emulator for supporting program operation on a heterogeneous OS platform or the form of emulating a hook API, this scheme is not suitable for the implementation of a hook layer.
For the above-described reason, a conventional implementation is made such that a hook layer manages addresses used to call hook target API original functions. Additionally, the hook layer is prevented from being influenced by the hook by additionally using auxiliary methods. Here, the programming scheme for implementing a hook layer in the form of managing addresses used to call hook target API original functions varies depending on the technique of the API hook. First, there is a method of performing no hook on modules that belong to a hook layer and lower layers by performing a hook exception on a unit runtime module, such as *.exe or *.dll, that belongs to the hook layer when a function address that is called by an API caller, such as an IAT hook, is modified. A condition that is required in this method is that runtime modules that belong to a hook layer should be accurately identified. Next, there is a method of configuring a hook layer by causing all runtime modules belonging to a hook layer to know addresses used to call the original functions of a hook target API and to call appropriate addresses at a program step. In this method, all runtime modules that belong to the hook layer should know the accurate list and addresses of APIs in which hooks are being performed.
However, in reality, there is a problem in that it is not easy to implement management/programming that can fulfill requirements for configuration of an ideal hook layer. As an example, when module-based exception handling is performed, there is a strong possibility of a runtime module that is made to belong to a hook layer because of the extension of functionality not being fixed. For this reason, a problem with the configuration and update management of the hook layer is inevitably magnified. Furthermore, since the information of an exception module based on the configuration of an OS and the configuration of main and lower versions may be added, the verification of the patches of an OS becomes very important. Meanwhile, the scheme for directly calling the addresses of an original API is problematic in that it is difficult to provide support to a module, the modification of the source of which is impossible, such as a third party module inside the hook layer, and, in particular, in that a program becomes complicated in proportion to the number of hook target APIs.
In order to solve the above problem, exception handling has been hitherto performed using the path of a file image, the pattern of a file image and the pattern of the issuer of an electronic signature depending on the decision of a developer. In this case, the burden of accurate list management is reduced, but the burden of the addition of a new module and the burden of the management of a list of a third-party module and the lower layer modules of a hook layer still remain.